Automated Vision-based Recognition of Construction Worker Actions for Building Interior Construction Operations Using RGBD Cameras

In this paper we present a novel method for reliable recognition of construction workers and their actions using color and depth data from a Microsoft Kinect sensor. Our algorithm is based on machine learning techniques, in which meaningful visual features are extracted based on the estimated body pose of workers. We adopt a bag-of-poses representation for worker actions and combine it with powerful discriminative classifiers to achieve accurate action recognition. The discriminative framework is able to focus on the visual aspects that are distinctive and can detect and recognize actions from different workers. We train and test our algorithm by using 80 videos from four workers involved in five drywall related construction activities. These videos were all collected from drywall construction activities inside of an under construction dining hall facility. The proposed algorithm is further validated by recognizing the actions of a construction worker that was never seen before in the training dataset. Experimental results show that our method achieves an average precision of 85.28 percent. The results reflect the promise of the proposed method for automated assessment of craftsmen productivity, safety, and occupational health at indoor environments. INTRODUCTION Activity analysis, the continuous and detailed process of benchmarking, monitoring, and improving the amount of time craft workers spend on different construction activities can play an important role in improving construction productivity, safety, and occupational health. As a workface assessment tool, activity analysis examines the proportion of time workers spent on specific construction activities. Combination of detailed assessment and continuous improvement significantly differentiates activity analysis from work sampling and can provide recommendation for activity monitoring, improvements, and improvements applicability (Gouett 2011, CII 2010). In recent years, many companies have experienced the benefits of activity analysis and are now proactively working towards implementing it in their projects (ENR 2011). Despite the benefits of activity analysis, the accurate and detailed assessment of work in progress requires an observer for every construction activity, which can be prohibitively expensive. In addition, due to the variability on how construction tasks are carried out, or in the duration of each activity, it is often necessary to record several cycles of operations. Not only are traditional time-studies labor intensive, but 879 Construction Research Congress 2012 © ASCE 2012